在许多情况下，有必要通过观察时间序列监视复杂的系统，并确定何时发生异源事件，以便采取相关的动作。确定当前的观察是否异常是具有挑战性的。它需要从历史数据中学习动力学的外推性概率模型，并使用有限数量的当前观察结果来进行分类。我们利用长期概率预测的最新进展，即{\ em Deep概率Koopman}，构建了一种在多维时序数据中对异常进行分类的通用方法。我们还展示了如何利用具有域知识的模型来减少I型和II型错误。我们展示了我们提出的关于全球大气污染监测的重要现实世界任务的方法，并将其与NASA的全球地球系统模型集成在一起。该系统成功地检测到由于COVID-19锁定和野火等事件而导致的空气质量异常情况。

Algorithms that involve both forecasting and optimization are at the core of solutions to many difficult real-world problems, such as in supply chains (inventory optimization), traffic, and in the transition towards carbon-free energy generation in battery/load/production scheduling in sustainable energy systems. Typically, in these scenarios we want to solve an optimization problem that depends on unknown future values, which therefore need to be forecast. As both forecasting and optimization are difficult problems in their own right, relatively few research has been done in this area. This paper presents the findings of the ``IEEE-CIS Technical Challenge on Predict+Optimize for Renewable Energy Scheduling," held in 2021. We present a comparison and evaluation of the seven highest-ranked solutions in the competition, to provide researchers with a benchmark problem and to establish the state of the art for this benchmark, with the aim to foster and facilitate research in this area. The competition used data from the Monash Microgrid, as well as weather data and energy market data. It then focused on two main challenges: forecasting renewable energy production and demand, and obtaining an optimal schedule for the activities (lectures) and on-site batteries that lead to the lowest cost of energy. The most accurate forecasts were obtained by gradient-boosted tree and random forest models, and optimization was mostly performed using mixed integer linear and quadratic programming. The winning method predicted different scenarios and optimized over all scenarios jointly using a sample average approximation method.

Few-shot methods for accurate modeling under sparse label-settings have improved significantly. However, the applications of few-shot modeling in natural language processing remain solely in the field of document classification. With recent performance improvements, supervised few-shot methods, combined with a simple topic extraction method pose a significant challenge to unsupervised topic modeling methods. Our research shows that supervised few-shot learning, combined with a simple topic extraction method, can outperform unsupervised topic modeling techniques in terms of generating coherent topics, even when only a few labeled documents per class are used.

The number of international benchmarking competitions is steadily increasing in various fields of machine learning (ML) research and practice. So far, however, little is known about the common practice as well as bottlenecks faced by the community in tackling the research questions posed. To shed light on the status quo of algorithm development in the specific field of biomedical imaging analysis, we designed an international survey that was issued to all participants of challenges conducted in conjunction with the IEEE ISBI 2021 and MICCAI 2021 conferences (80 competitions in total). The survey covered participants' expertise and working environments, their chosen strategies, as well as algorithm characteristics. A median of 72% challenge participants took part in the survey. According to our results, knowledge exchange was the primary incentive (70%) for participation, while the reception of prize money played only a minor role (16%). While a median of 80 working hours was spent on method development, a large portion of participants stated that they did not have enough time for method development (32%). 25% perceived the infrastructure to be a bottleneck. Overall, 94% of all solutions were deep learning-based. Of these, 84% were based on standard architectures. 43% of the respondents reported that the data samples (e.g., images) were too large to be processed at once. This was most commonly addressed by patch-based training (69%), downsampling (37%), and solving 3D analysis tasks as a series of 2D tasks. K-fold cross-validation on the training set was performed by only 37% of the participants and only 50% of the participants performed ensembling based on multiple identical models (61%) or heterogeneous models (39%). 48% of the respondents applied postprocessing steps.

Predicting the future development of an anatomical shape from a single baseline is an important but difficult problem to solve. Research has shown that it should be tackled in curved shape spaces, as (e.g., disease-related) shape changes frequently expose nonlinear characteristics. We thus propose a novel prediction method that encodes the whole shape in a Riemannian shape space. It then learns a simple prediction technique that is founded on statistical hierarchical modelling of longitudinal training data. It is fully automatic, which makes it stand out in contrast to parameter-rich state-of-the-art methods. When applied to predict the future development of the shape of right hippocampi under Alzheimer's disease, it outperforms deep learning supported variants and achieves results on par with state-of-the-art.

The capture and animation of human hair are two of the major challenges in the creation of realistic avatars for the virtual reality. Both problems are highly challenging, because hair has complex geometry and appearance, as well as exhibits challenging motion. In this paper, we present a two-stage approach that models hair independently from the head to address these challenges in a data-driven manner. The first stage, state compression, learns a low-dimensional latent space of 3D hair states containing motion and appearance, via a novel autoencoder-as-a-tracker strategy. To better disentangle the hair and head in appearance learning, we employ multi-view hair segmentation masks in combination with a differentiable volumetric renderer. The second stage learns a novel hair dynamics model that performs temporal hair transfer based on the discovered latent codes. To enforce higher stability while driving our dynamics model, we employ the 3D point-cloud autoencoder from the compression stage for de-noising of the hair state. Our model outperforms the state of the art in novel view synthesis and is capable of creating novel hair animations without having to rely on hair observations as a driving signal.

Binary neural networks are the extreme case of network quantization, which has long been thought of as a potential edge machine learning solution. However, the significant accuracy gap to the full-precision counterparts restricts their creative potential for mobile applications. In this work, we revisit the potential of binary neural networks and focus on a compelling but unanswered problem: how can a binary neural network achieve the crucial accuracy level (e.g., 80%) on ILSVRC-2012 ImageNet? We achieve this goal by enhancing the optimization process from three complementary perspectives: (1) We design a novel binary architecture BNext based on a comprehensive study of binary architectures and their optimization process. (2) We propose a novel knowledge-distillation technique to alleviate the counter-intuitive overfitting problem observed when attempting to train extremely accurate binary models. (3) We analyze the data augmentation pipeline for binary networks and modernize it with up-to-date techniques from full-precision models. The evaluation results on ImageNet show that BNext, for the first time, pushes the binary model accuracy boundary to 80.57% and significantly outperforms all the existing binary networks. Code and trained models are available at: https://github.com/hpi-xnor/BNext.git.

Density based representations of atomic environments that are invariant under Euclidean symmetries have become a widely used tool in the machine learning of interatomic potentials, broader data-driven atomistic modelling and the visualisation and analysis of materials datasets.The standard mechanism used to incorporate chemical element information is to create separate densities for each element and form tensor products between them. This leads to a steep scaling in the size of the representation as the number of elements increases. Graph neural networks, which do not explicitly use density representations, escape this scaling by mapping the chemical element information into a fixed dimensional space in a learnable way. We recast this approach as tensor factorisation by exploiting the tensor structure of standard neighbour density based descriptors. In doing so, we form compact tensor-reduced representations whose size does not depend on the number of chemical elements, but remain systematically convergeable and are therefore applicable to a wide range of data analysis and regression tasks.

在本文中，我们考虑了通过风险最小化监督学习中变异模型的问题。我们的目标是通过双层优化和通过算法展开对学习变异模型的两种方法进行更深入的了解。前者将变分模型视为低于风险最小化问题的较低级别优化问题，而后者将较低级别优化问题替换为解决上述问题的算法。两种方法都在实践中使用，但是从计算的角度来看，展开要简单得多。为了分析和比较两种方法，我们考虑了一个简单的玩具模型，并明确计算所有风险和各自的估计器。我们表明，展开可能比双重优化方法更好，而且展开的性能可以显着取决于进一步的参数，有时会以意外的方式：虽然展开的算法的步骤大小很重要，但展开的迭代数量只有很重要如果数字是偶数或奇数，并且这两种情况截然不同。

安全关键系统通常在调试之前进行危害分析，以识别和分析操作过程中可能出现的潜在危险系统状态。当前，危害分析主要基于人类的推理，过去的经验以及清单和电子表格等简单工具。增加系统复杂性使这种方法非常合适。此外，由于高成本或身体缺陷的危险，基于测试的危害分析通常不适合。对此进行的补救措施是基于模型的危害分析方法，这些方法依赖于正式模型或模拟模型，每个模型都具有自己的好处和缺点。本文提出了一种两层方法，该方法使用正式方法与使用模拟的详细分析结合了详尽分析的好处。首先使用监督控制理论从系统的形式模型中合成了导致不安全状态的不安全行为。结果是输入到模拟的输入，在该模拟中，使用域特异性风险指标进行了详细的分析。尽管提出的方法通常适用，但本文证明了该方法对工业人类机器人协作系统的好处。